D0CDH3B3! BBSDIIS 



ED 133 333 Tfl 005 566 

JIUTHOE SpaneCi Stmmn D. 

TITLE What iDfisrencQS are Allowable Sdth a SignificaDt F in 

Eegressloii Amalysls'i 
PUB DAM Ikpx 76] 

NOT! I5p*; Iap€C pceaanted at the Annaal fleeting of the 

American laucatlonal Heseacch Association {60th^ San 
FrancigQQ g Cali f ornia , 4pril 19*23 , 1 976) 

IDES PBXCl Plus Postage, 

DESCRIFIOES *inalyals of ?a:ciaBca| ^Hypothesis Tastings ^Multiple 

Eagjcessiou Analyaisi *Test3 of Significance 
IDENTIFIBBS StatistlaaL Inf^cance 

^BSTOACr 

The InJereaces allowable vith a sigtiificast F in 
Eagrassion analyals ^tm discvss^a* Incl^d^fl in this discussion are 
the effects of spacificltj of tie zes€aE'Ch bypothasis^ incorpoca tion 
Qf covariates^ directicaal hypotheses i and tie Danipalatlon of 
yariatlas on the intarprt tation of aignif icaiice for such purposes a a 
causal and diractiomal i^nferenc^s* The position Is takGii that tht 
rasearah hypothesis aictates the variables to b« tested amd h€nat 
included ia tha ragressioii meaals* If the Milafclas ha^va heen 
ma]niptilatadf then caasal iaferejice^ can potemtially be nads* If 
covariatas hava beeii included iii the analysis, theji they shouia be 
incladed in tha inferaact. If tie research hypothasis specifi^a a 
directional expectation^ then a airectional conclusion is war ranted^ 
PC) 



^ Dociiaeiits acgiaired by IHIC inclttda aany iDforoal unpublish^a ^ 

* inaterials mot aFailabls frqn Qthar soaxcas, BIIC makes e^ery effoxt ^ 
^ to obtain the best oopy available, nevertheless^ iteas of aacginal ^ 

* reproducibility ara often encounterea and this affects the quality ^ 

* of tha nicroficha and hardcopy reprcauctions IRIC makes availaile ^ 

* vLbl the EBIC Docuttant leproducticn Sertica (EBRS) * EDRS is aaot ^ 
^ responsible for tha gttillty of the original docunent* EeprodQctioms ^ 
^ supplied by BDBS are the best that can he aade from the originals ^ 



EKLC 



/, 2. / 



What Irifefences are allowable with a 



slgnificart F in regrtssion analysis? 



Steyen Spaner 



University of Mfssourl - St* Louis 



us DepftsrMfiwTQFNiiLTM, 

THIS OaCUMSNT NmS BEgN R EP^O- 
DuCED EXACTLY AS R tCElVED FROM 
THE SSONDR ORfiANUATlONOftlGiN- 
ATiNO iT POINT! OF V>EWOi? QPII*j|ON| 
STATED DO NOT NiCiSSARlL^ RepRi^ 
SENT OFFICIAL NATIQMA.L INITITU Ti QF 
iOUCATiON POSITION POLICV 



A paper presented at the symposium '*Issues in the use, interpretatloni 
and teaching of multiple linear regression" sponsored by the Special 
Interest Group on Multiple Linear Regrission and the Anarlcaii 
Education Research Association Convention , 



San Fran CI SCO 5 1976 





ERIC 



What infirences are anowable v/ith a 
significant F in regrisslon analysis? 

Steven D. Spaner 
University of Missouri - St. Louis 

f^ultlple linear rigression (MIR) procedures (Wainer, 1976 ) and the 
linear fficdel tn general (Brown^ 1975) have come under attack in the 
most recent manths, And^ sad to say many of the points raised are cogent 
criticisms indeed. However, none of the remarks are foreign to the mem- 
bers of SIG-WLR or the readers of Multiple Linear RegressiDn Viewpoints. 
Biit, perhaps these admonitions from cur brethren wouAd be less audible 
if the topics to be discussed here today were fnore untversany heard. 
WJiat appears to be needed is a better understanding of MLR: its foun-^ 
detions, its applications, and its ramifications* My presentation ad- 
dresses some of these misunderstandings. 

We calculati regression coefficients, we ditermine R^s , we compute 
F statistics 3 but do we know what it's all about? If the audience 1s 
composed of Ward-Kelly-McNeil proteges then the word ''assuniptions" elfdts 
a dlscountfng laugh; but there are those who read a list of assumptions 
in a statistics text and live them or die by them, I believe an 
exariii nation of the foundations of MLR would do both the jester and the 
Jouster some good. 

Drawn' ng froni a variety of sources (Ward & Jennings^ 1973^ Llndquist, 
1953j McNelU Killj, & McNtiU 1975^ Snedecor, 1956; Young 8 Valdrnari, 
1972; and Glass & Stanley^ 1970} the fonowing list of assumptions have ■ 
been iden tiffed for the F statistic: 
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1 . The F statistic must have been generated from randomly selected 
and Independent entities or criterion measures, 

2. The vaHdnce of thi criterion measures within each population 
subgroup irust be equal (hoinogenelty of variance), 

3, The distribution of the criterion mmsurm in each population 
must be normal » 

Specific authors hive added to or restated in other words the above 
list. The on! J rneaningful change would occur 1n the case where a 
cov^ariance analysis had been perfornied and the F statistic generated from 
these data* In such an Instance the above assuiiiptt ons appl^ to the ad- 
justed criterion measiures and a fourth assumption Is added: 

4, The regressions of the criterion measurts onto the covariate(s) 
are equal for each population subgroup (fioiBogenelty of regression). 

Before t summarily dismiss these P ratio assiiniptions with an im- 
pressive list of citations^ It should be pointed out that another list 
of assumptions are of concern. While the F dtstrlbutlOT is the theoreti- 
cal sampling d1stribut1oni the overall cal culatiort technique is regrfsslori 
v/hich 1s syn&nym^us with correlation (In point of facta Galton postulated 
his 'Mav/ of universal rigression'' before Pearson developed the index of 
correlatlon/shlpy) . Hence we must also acknowledge the set of assumptions 
underlying regression* These assumptions are essentlaiythe characteristics 
of the bivariate normal distribution: 

U K scores f disregarding Y scores p are norma 11 y distributed* 
2^ Y scores^ disregarding X scores^ are normally distributed* 

3. The Y scores for each X score are normany distributed with a 
conmon variance (CTy^^) , 

4. The X scores for each Y score are normany distributed with a 
comffion variance iO'J'y) . 



5* The means of the Y score distributions for each X score fall 
on a straight line. 

Since we are here to discuss rriult iple regrisslon th€ above assumpt'ions 
must be expanded to address tfie mul tiyariate norfnal distri butloni but I 
wm leave that task to the reader^ the features are essentially^ the same, 
Hov^iver^ the expansion brfngs up my first point v/ith regard to allowble 
1 nf erences , 

Snedicor (1956) has classified multiple regression into tw 
basic models: Model I - the values of X are considered flxeds that Is^ 
chosen by the 1 nvestlgatori onl^ the Y \falues or the criterion 1s a 
random* normally distributed variables and Model 11 - the values of X 
are not selectecls Individuals are randonnly selected leaving the valiies 
of every variable measured on the individuals available to chance; that 
is^ a random sample Is drawn frofn a multivariats norrnal populatior. 
The first model is e J(enip 1 1 f i ed by the ^ posj factQ research design and 
designs- in W'hich treatiiients are not randomly assigned. The second node! 
is ntare In the tradition of the experiniental design. Investigations 
utilizing model I oftep take the farm of covariance analyses which call 
for the 4th assiiinption to the F statistic and call far a limitation on 
the generalisations; i,e,^ the population of adjusted y scores, Hovevers 
statistical control shculd not be yiewed as limiting since it allows 
one to study the actual situation instead of one that has been artificially 
producad by experimental control, 

Lindquist (19S3J has observed that in educational research the ap- 
plication of model tv/o is often amended to random asslgnriient of treatinents 
instiad of randoin assignment of subjects to treatmants. The reasons 
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are obvious and the amendment Ingeniously adaptive; hov/ever, it must be 
understood by the researcher that his population has changed. The pop- 
ulation 1s no longer Individuals who are potent ially avanable for randofn 
selection but instead Intact groups who are available for random assignnient 
©f treatnients , 

Lest you begin thinking that assumptions are all this paper is going 
to address let me put your thoughts at rest. A number of investigations 
have dealt v^ith the f distribution assumptions and their -violation 
(Norton^ 195Zj BonneaUi 1960, 1963; Young S Veldman, 1 963 | Pearson, 1931j 
Bo^ S Andersonj 1955). The summarized conclusion from thtst investigations 
Is that there 1s no appreciable effect on the accuracy of the F test 
frofn nonnortnality and if sample sizes are equaU heterogerelty of variance 
has a negligible if feet. The only apparent serious violation that can 
be conmitted Is failure to randonily select Independent entitles or 
measures (Glass & Stanley, 1970 However , surprisingly, tNre are no 
implrlcal investigations of this tenet. Furthermore , there is a dearth 
of inquiry into tlie effects of violation of the hoinogeneity of regression 
assujnption and the list of postulates for regression and correlation. 
From the fev^ statements offered on these topics (Snedecora 1956; Vasu & 
ElniDrep 1975} it appears that once again normality is a mute issui^ but 
that dependence of observations (ri*95) can cause disruption of accurate 
calculations, Snedecor (1956) has recommended the elinil nation of one of 
the pair of Xs with correlation greater than .95 (based on a redundancy 
interpretation). However^ McNeil and Spaner (1971) ha v& made a case 
for a more judicious examination of such a recorrBiiendatlon* especially 
as it would apply to nonlinear problems. 
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Having fulfilled my obligation to the ccnservati ves ; having ac- 
knowledged the assufnptions we are operattng under in NLRi I will now turn 
to the topic sentences "What inferences are allowable with a significant 
F in regression analysis?" The first and foreniost limitation on our 
inferences Is the research tool; i.e., the t^pe of regression teGhnique, 
stepwise or hypothesis testing regression. Ny collaague (NcNell, 1976) 
has addressed himself to this matter so I will make onlj^ a brief reniark* 
The calculation of an F statistic carries with it the implication that a 
comparison is deslredi a decision is to be maide, and that an inference 
will follow* All three of these activities suggest that a cofnparlson 
hypothesis has been adopted (a null hypothesis) and an alterratlye hy- 
pothesis will be accepted should the comparison hypothesis be found un- 
tenable, Hypothesis testing regression fits this research fontiat precisely 
a null hypothssis is formulated - the restricted model as Bottenburg 
and Ward (1963) tagged it^ and an alternative hypothesis 1s proposed - the 
full model in Texas terminology (this form of labeling has been princi- 
pally associated with University of Texas faculty and graduates). A 
significant F statistic, in the "grand tradition*'i calls for rejection 
of the null hypothesis (restricted fnodel) and aeceptance of the alternative 
hypothesis (fiin model ). 

Let us look nov/ at stepwise regression. Stepwise comes in two 
forms: ascending and descending. Ascending stipwise regression adds 
variables to a null set until a new set is created which has maKimum 
predictive efficiency (according to some criterion}, ^Descending stepwise 
regression operates in just the reverse; from a defined set of variables^ 
variables are refnoved which least add to efficient prediction (until some 
*'stop" criterion 1s obtained). ExainlnatiQn of these two techniques in 
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relation to an F test leaves the researcher with a "loose end". In the 
case of ascending stepwise, the null hypothesis is known, it's .a. modal of 
the criterion grand mean. But, the alternative hypothesis 1s unknown; 
hence, no decision can be contemplated and no inferences entertained that 
are not sainple generated. Likewise, 1n descending stepwise the alternative 
hypothesis is known, but we must wait for the "marvelous toy" to tell us 
what our coniparison (null) hypothesis is. Therefore, if Inference and 
generalizations are to be allowed upon significant F tests we must con- 
duct hypothesis testing regression analyses, 

A second limitation on our allowable Inferences relates to the research 
hypothesis. Many of the more Interesting and pertinent questions In edu- 
cation call for the statistical control of variables that are practically 
or explicitly beyond experimental control, Indeed, one of the attractive 
features of MLR is the ease with which covaflance analysis can be con- 
ceptualljy as well as operationally understood (Williams, 1976). And while 
we have a number of admonitions against causal interpretations in ex pos t 
facto (basically correlational) studies (Campbell & Stanley, 1969, McNeil, 
Kelly, & McNeil, 1975-, Newman, et. al_. , 1976); we would do well to remember^ 
"... that in niany cases statistical control is more to be desired; the 
actual situation is studied Instead of one artificially produced, the 
observations are extended over a greater range, thus broadening the foun- 
dation for Inference, and in the end one has knowledge of the variation of 
two [or more] quantities instead of one, together with the relationship 
betv^een them." (Snedecor, 1956, p. 146) However, covarlance analysis 
it win be remembered Imposes a 4th assumption on the F distribution - 
homogeneity of regression. Ironically, McNeil and company (1975, p. 131) 
have shov^n us how easy it is to make a test of this assumption with MLR 
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(i.e,, interact the mernbership variables v/ith the coyaHates and tast the 
equality of the coefficients); yet, users and teachers (Wmiams, 1976) 
seem to be ynfamniar with this easy but necessary test* Hence » Inferences 
based on significant F tests of covarlance analyses will be rendered in- 
accurate If not invalid without the hofiiogeneity of regression test* 

Indeed the value of and need for interaction tests has been grossly 
undereniphaslzid in MLR studies. I suspect that this p^hinomenon arises out 
of a misunderstanding t perhaps even feari of a signlfiMfnt interaction 
finding. Trues a significant interaction hampers the Interpretation of 
fnain effects, but the positive view is that a significant F test of 1nter» 
action tells us how to appropriately liniit our generalizations (Glass & 
Stanley^ 1970). 

Another Issue related to hypotheses and having a tearing on our in- 
ferences with a significant F statistic is that of directionality. Without 
going into the rudiments of sampling theory i I will suinniarize the direction- 
ality vs, nondi recti onal ity decision as one which doubles your chances 
of rejecting the null hypothesis, vf you hypothesize In the right direction, 
It*s that last phrase that seems to be overlooked by mainy researchers. 
There Geems to be a preponderance of cautious research hypotheses (non- 
directional) and bold research conclusions (directional)* Let me give some 
guidance as to this choice of hypotheses as it relates to MLR, It must 
be refnernbered that regrassionp especially least squares^ fegressioni is 
basically a curve fitting technique (Lewis, 1960| Snedecor, 1956). This 
being the case there are only three aspects of a curve thSit are mani- 
pulatable: 1 ) the point where the curve Intersects with some reference 
axis (intarcipt point), 2) the rate of rise of the curve (the slope^ which 
is an indicator of relationship), and 3) the number of Inflection points 
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in the curve (this 1s governed by the expontntiatlon factor). Of these 
three factors only one is avanable for nondirectlonal hypotheses^ the 
cornparison of Interctpt points. The weightings or regression coefficients 
that identify level of Intercept are indicative of group means. And 
direction of differences between group means can be unknown such that a 
nondi recti onal hypothesis is conceivable. Of course^ each researcher 1s 
characterized by his or her own ri sk-taking-behavlors but there are strong 
arguments for stating all research hypotheses as directional hypotheses 
(McNeil, Kellyp McNeil, 1975), 

The other two aspects of a curve call for directional hypotheses by 
the very nature of their source. The source of rate or slope changes is 
the addition of information into a model* The worse that can happen is 
no change; hopefully the added Information will enhance the relationship 
of the predicted scores with the actual scores (1,e. increase ^y)* And* 
since our yardstick of meastirimint is error sum of squaras m cannot 
generate more prediction error by Including Information which is related 
to the criterion. We can, however, possibly reduce the error by the in- 
clusion of new information (Ward & JanningSj 1973; Snedecori 1956; McNeil, 
Kelly, & McNeil, 1975). 

The same mathematical truths hold for hypotheses about Inflection 
points. That is^ in hypothesizing models that are to fit nonlinear data^ 
the only possible hypothesis 1s a directional hypothesis. Inclusion of 
exponentiated variables allows the best fit line to bend and turn with the 
data thus reducing the sum of squared deviations from the line* Howevet* 
if the iKponentiated variables do not create a better fit, they do not 
create more error, they simpl^^ take up degrtes of freedom withQUt error 
reduction and riceive lero or 1 ow weightings. 

Htnci, the point to be made is that only directional hypotheses are 
being tested with a significant F in two of the types of regression 
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hypothesis^ whether the researcher has stated them as such or not* Only 
in the statemint »f hypotheses about intercepts (group mean differences) 
Is thire the poteritlal for a nondlrectlonal F test and there is great 
suspect of a researcher wiho v^ould not have some expectation of the direction 
of mean differences. 

Returning momeintaHly to a point made about nonlinear fits of data; 
there are recurring wlls for the deyelopment of nonlinear hypotheses In 
educational research^ the TOst recent coming from Brown (1975), Being 
myself one of the mr© ancient heralds of this Idea (McNeil & Spaner, 1971) 
and knowing that I was rot the first to send smoker I find It a puzzlement 
as to why there aifi not more nonlinear hypotheses in educational research* 
I come up with two possible reasons* ones the "Pandora's box" fear, 
and two the fear of v^iolatlon of assumptions. With regard to the first 
reason* Pandora's boK, It Is true that there is an Infinite set of non- 
linear terms. How^vtri* biwariate plots of sample data can narrow down 
the field of fruitful hypothases quite well* Additionanys there are com- 
puter aids to "snoop aroyirid" In sample data and arrive at sora tenable 
hypotheses (Automatfa interaction Detection-Version 4 by Koplyay^ Gott, & 
Elton, 1973), 

The second fear,, Vfiolatton of assumptions, must refer to the line- 
arity of regrisslon assumption since the first portion of this paper has 
indicated that noraalfty mi homoscedasticity are not critical* So let's 
examine the meaning: of this assumption: it states that the means of all 
sampled populations Hi on a stra^jht line. But what is the effect of 
this assumption? It allows us to infer for populations not sampled* We 
assume the populatior^ mmn for unsampled populations falls on the sampled 
regression line. And^ this is a valuable principlei indeed^ for without 
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it we have no prediction. However, does this regression Una have to be 
straight? If it does, then there are a number of logarithmic transfor- 
mations of curve line functions that will bring the curved best fit line 
back into line (LewtSt 1960). But all this manipulation 1s unnecessary; 
it is simply a rescaUng process. It's a recognition that the measuring 
devices we use are not necessarily monotonic. It^s a recognition^ as 
Pohlman j Newman (1973) have suggested, that the assuinption of recti- 
linearity has not bean met- But more than that, it's 8 recognition that 
the assumption can be met 1f a curved best fitting line is used. McNeil 
& Kelly (1970) and Ward & Jennings (1973) have addressed themselves to 
this issue and have suggested that the investigator attempt to express 
functional relationships in data first and not worr^ about v/hether con- 
ditions or assumptions are met. In other words^ If ft works, use it. 

As a concluding remarki it should be pointed out that "what Inferences 
are allowable with a significant F in regression analy:ais "are zero if 
the r2 of the regression model is not practically significant J' Ward & 
Jennings (1973) have Indicated that very large samples can produce 
statistically significant F statistics with very littlt* practical con- 
sequences. Scientific rigor and the scientific process produce significant 
results, statistics simply apply a probability level to your potential 
error in judgment. Focus should be placed on 1) randofni and independent 
sampling, 2) clear and precise statements of research hj»potheses, 3) 
construction of appropriate models to test the hypoth^sasj 4) examination 
of R values, and 5) allowable inferences from significant findings* 
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